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Abstract > 

Much research on dexterous robot hands has been aimed at the design and control problems as so* 
dated with their autonomous operation, while relatively little research has addressed the problem 
of direct human control. It is likely that these two modes can be combined in a complementary 
manner yielding more capability than either alone could provide. While many of the issues in 
mixed computer/human control of dexterous hands parallel those found in supervisory control of 
traditional remote manipulators, the unique geometry and capabilities of dexterous hands pose 
many new problems. Among these are the control of redundant degrees of freedom, grasp sta- 
bilization and specification of non-anthropomorphic behavior. This paper^vifhgive an overview 
of progress made at the MIT AI Laboratory in control of the Salisbury 3 finger hand, including 
experiments in grasp planning and manipulation via controlled slip. We also suggest how we 
might introduce human control into the process at a variety of functional levels. 




Introduction 

Robot hands have the potential to increase the utility of teleoperator or telerobotic systems 
systems by a large amount. By robot hands we refer to the emerging generation of complex, 
articulated mechanical hands designed for computer and/ or human control. While such 
complex devices have yet to see application outside of research environments, significant 
progress has been made in the development of manipulation models, hand control and 
mechanics (Mason and Salisbury][Jacobsen]. Such capacities as secure grasping, controlled 
grasp force and impedance, controlled slip, cartesian control of fingertip and object motions 
and a lisp-based language for hand control have been demonstrated [Salisbury, Brock and 
Chiu]. 

Autonomous, program controlled hand systems represent one extreme of the range 
of command modes in which we expect the hands in telerobotic systems to operate. The 
other extreme is typified by the purely mechanical remote hands built by [Jameson 87] 
and proposed by [ADL]. These devices utilize mechanical linkages to conduct forces and 
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motions imposed on the master links to the output hand. Ultimately we envision systems 
which will take advantage of both direct and preprogrammed control of hands, permitt ing 
both human intervention in programmed tasks and computer augmentation of manually 
controlled operation. The motivation for using such complex end effectors lies in improve- 
ment of manipulative dexterity and adaptability that we expect to achieve. In order to 
assess this potential against the costs of increased complexity it is important that one 
understand the unique functional capabilities offered by complex articulated hand. 

Hand Functions 

The functions which hands are required to perform can be divided into several areas. 
Prehensile, manipulative and sensory functions are perhaps the most common. In human 
experience they often occur in combination with each other in a mixture of exploratory and 
purposeful motions. In performing complex tasks humans simultaneously employ all three 
functions to verify, guide and plan the sequence of operations performed. In the robotic 
world we are not yet so lucky as to be able to smoothly integrate these complex functions 
into a synergy of action; current robotic systems employ a rather loose coupling of their 
prehensile, manipulative and sensory functions, both structurally and operationally. 

Prehensile capability implies the capacity to hold object in some controlled state rel- 
ative to the hand. This requires that the grasp allow exertion of sufficient restoring forces 
on an object to hold it against arbitrary disturbance forces. Secure grasping requires at 
least that there be sufficient dimension to the collective effect of forces acting through all 
the contact points to span the’ space of disturbance forces which may act on the object. 
For spatial motion of rigid objects this is six. The constraints imposed by contacts can be 
divided into structural constraint and frictional constraint. A grasp which employs more 
structural constraints will be more secure against large disturbance forces; frictional con- 
straints will cease to be active if disturbance forces become too large in certain directions. 
The dimension of the net forces acting on the grasped object can be no greater than six, 
therefore if extra freedoms in force exertion exist, (i.e. more active joints) then the hand 
has the freedom to adjust the internal forces. These internal forces may be varied without 
changing the net force acting on the body. They do oermit, within certain limits, changing 
the direction and magnitude of forces acting through individual contact points. This free- 
dom in allocation of grasp forces permits some degree of optimization of the grasp. One 
potential criterion is minimizing the reliance of a grasp on friction constraints. See also 
[Cutkosky], [Kerr] and [Jameson]. 

Manipulation of objects with a hand can take several forms. It may simply hold 
an object securely relative to the wrist and rely on the rest of the arm for imparting 


352 



motion. This is typical of most robots and of humans moving relatively large objects. 
Motion of smaller objects can be achieved with the fingers alone. Objects held in the 
fingertips can be moved and rotated short distances without breaking or sliding finger 
contact. Larger motions within the hand require controlled slipping and repositioning of 
fingers. In manipulating very large objects the hand may recursively act as a large fingertip 
in concert with other cooperating hands. 

Sensing includes a number of functions ranging from simple measurement of the mech- 
anism’s position, velocity, acceleration and force states, detection the characteristics of 
contacts with objects such as location and force distribution to the measurement of ma- 
nipulated object characteristics such as curvature, texture and temperature. 


Dexterity 

Dexterity then, is the integration of all these and more capacities into a higher level of 
competence. It is a quality of manipulative capability which implies a degree of skill 
and deftness of operation. These in turn, depend on the quality of motion, sensing and 
control. If we look at examples of increasingly dexterous motions the distinction between 
prehension, manipulation and sensing begins to blur. Sensing the shape of an object 
may require manipulation of it to detect its contours. Holding an object may require 
adjusting to a more favorable grasp. And, moving an object may require a sequence 
of regrasping. Humans easily integrate these functions to perform wonderfully complex 
operations. A high degree of mechanical functionality, rich sensory input and intellectual 
capacity combine to produce human dexterity. Dexterity in the machine sense is still in 
its infancy and suffers from limited functionality in all three areas. 

Machine dexterity necessitates that a high degree of mechanical functionality be avail- 
able, independent of whether the commands originate with a human or a computer. In 
a telerobotic system the hands and arms will be controlled by combinations of human 
and computer command, but the net dexterity can be no greater than the mechanical and 
sensory capability of the system. These elements of machine dexterity can be divided into 
several broad categories. 

Kinematics. There must be sufficient contacting surfaces to impart the required degree 
of constraint to manipulated objects. There must be sufficient degree of freedom of 
motion in the hand and fingers to impart required motions and forces to objects. The' 
joint ranges of motion and link proportions must be sufficient to achieve required 
grasps and motions. 
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Dynamics. The mechanism must be fast enough and strong enough to do the required 

work. It must have sufficient resolution of motion and force exertion to perform the 

required tasks. 

Sensing. The type, accuracy, sensitivity and resolution of sensing must be adequate to 

provide required perception of hand states and object interactions. 

The kinematics and dynamics of a hand limit the controllability of states of the manipulated 
objects. The sensory system limits the observability of task states. A mechanism lacking 
certain degrees of freedom my not be able to effect certain motions of a manipulated 
object; a mechanism lacking certain state information may not be able to detect particular 
motions. Thus the controllable and observable states permitted by a hand must overlap in 
the dimensions in which we want to achieve closed loop control. Beyond these requirements, 
a dexterous hand must be constructive, reliable and efficient - a tall order for mortal 

designers. 

The shortcomings in current robot dexterity can be alleviated to some degree by 
human intervention. Structurally, improvements can be made by redesign of the mech- 
anism to include required freedoms of motion and “active” surfaces. By active surface 
we mean surfaces on the hand (arm, body or whatever) that may be brought into play 
to contact and impart motion to manipulated objects. Functionally, such capabilities as 
force control, stiffness control, programmed reflexes, and programmed micro-tasks may be 
added. Improvement in mechanisms and servos will improve the quality of these functions. 
Intellectually, commands may be augmented by writing better and more “intelligent pro- 
grams, and/or a human may assume more direct control via analog input devices, ranging 
from switches to complex master devices. Perhaps the most difficult aspects of machine 
dexterity are the closely coupled intellectual and perceptual components. In performing 
truly complex tasks which require a high degree dexterity the demands for high level “un- 
derstanding’of task goals and progress is greatest. While increasingly more autonomous 
systems will be developed, it is at this high level that direct human intervention is of 
immediate value. 


Human Intervention 

The human may intervene in the telerobotic manipulation process in a number of ways. At 
the programming level this will be limited by our ability construct systems able to model 
tasks physics, automatically plan operations and react to unexpected situations. At the 
direct interaction level the human will be coupled though a master control device to the 
hand and arm control system. Such master devices are designed to measure motions and 
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forces imparted by the human on the device which are in turn used to generate commands 
for the remote hand. 

The motions the operator will be able to perform and the information he will be able 
to acquire will depend upon the mechanical and sensing qualit ies of the master device. In 
fact, the qualities required of the master mirror those of the remote hand. The human can 
directly control no more degrees of freedom of motion than present in the master device 
and may directly sense no more force or kinesthetic information than can be reflected to 
the master. Such direct control implies that the master must be a kinematic replica of the 
remote hand with a one-to-one correspondence between master and remote joints. Control 
of this type has been demonstrated with the Utah/MIT hand [Jacobsen] in a non-force 
reflecting mode. Even without the benefit of force reflection to the master reasonable 
dexterity can be achieved through visual feedback and the operator’s ability to anticipate 
required motions without feedback. The challenge of designing a force reflecting kinematic 
replica master hand of sufficient quality to be useful is a formidable one and suggests that 
we also consider reduced complexity master devices. 

A generalized master for controlling finger motions might consist of thimble-like re- 
ceptacles into which the operator’s fingers are placed. These would be supported by a 
linkage capable of following the finger motions and exerting forces upon them. This mode 
of control would not necessarily preserve the one-to-one correspondence between human 
and robot joint motions; it would require computer mediation to resolve these motions. It 
is perhaps most appropriate for very dexterous manipulation of objects held in the finger 
tips where we may abstractly view the robot fingers as a collection of force applicators. 

An even simpler master might be constructed in the form of a flexible handle. This 
deformable masterc oulcl consist of a handle with one or more articulated surfaces that 
the operator presses against during operation. It would be limited to reflecting the net 
force acting on the grasped object and perhaps one or two of the internal grasping forces. 
Movement of the robot fingers would depend more heavily upon prespecified and computer 
controlled coordination patterns. 

As we move from complex to reduced freedom master devices we must map commands 
from the master into the more complex motions of the robot hand. This may be done by 
assigning master motions to vectors of robot joint motions. Sensory information from the 
remote hand would have to be mapped back to the master. The choice of these mappings 
would depend upon what variables are most appropriate for human intervention and what 
variables may be assigned to computer control. In all cases the control of the hand would 
have to be integrated with the arm control. Any master for commanding finger motions 
would probably best be connected to the master device used for commanding arm motions. 


355 



This would tend to preserve the operator’s kinesthetic sense and help promote synergistic 
use of the hand and arm as a whole. 

As experience and capability with autonomous hands accrues, the functional capabili- 
ties developed will be able to assist the human operator to greater degrees. In the following 
section we describe some of our progress in autonomous robot hand control as a means of 
suggesting elements which ultimately may be integrated into a telerobotic system. 

Experimental Hand Environment 

As part of our group’s research in dexterous hand manipulation with the Stanford/JPL 3- 
Finger Hand (Salisbury Hand) we have developed the experimental environment described 
in [Salisbury, Brock and Chiu]. This hand, shown in Figure 1, has three fingers each with 
three active joints for a total of 9 degrees-of- freedom. It has been designed to have the 
minimum number of freedoms required for securely grasping and arbitrarily moving objects 
held in the fingertips as described in [Mason and Salisbury]. The control system employs 
the hierarchy of computers shown in Figure 2. There are 12 microprocessors at the lowest 
level for individual motor control. A VAX 11/750 is used at the mid-level for controlling 
finger trajectories and various monitoring functions. Figure 3 depicts the data structure 
used to describe actions at the joint level. It is essentially a linked list of joint space 
trajectory polynomials, with associated stiffnesses and condition monitors. Simple and 
rapid reacting to triggered conditions may occur at this level, including sending a message 
to the Lisp machine. A Symbolics 3600 Lisp machine is used for commanding cartesian 
motions of the hand, to react to triggered conditions and to provide a flexible user interface 
for new modules. Primitive capabilities for the system include position, force and stiffness 
control. The system permits commands and sensory information to be specified in joint, 
fingertip and grasped object coordinate systems. For reasons of efficiency and modularity 
the VAX trajectory and force control systems operate entirely in joint space. Cartesian 
and more abstract frames of reference are reached through the Lisp interface. 

The system has become a test bed for verification and development of higher level 
functions required for dexterous, autonomous operation. Among the capabilities that have 
been demonstrated are a contact resolving sensor, a grasp planner module and a controlled 
slip module. The contact resolving sensor developed by [Brock 84] and [Salisbury 84], 
shown in Figure 4, permits locating the position and orientation of a contact and well as 
the determination of the tangential (frictional) and normal (structural) components of the 
contact force. It has been used to map the contours of an object placed in front of the hand. 
The grasp planner developed by [Nguyen] has been used to determine and execute stable 
force-closure grasps on a 10cm cube. The planner determines acceptable grasping surfaces 
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and specifies appropriate finger tip stiffness for polyhedral objects. Because the planner 
uses only the physics of grasping to guide it and because the hand has a range of motion 
beyond anthropomorphic, the planner identifies a number of unusual (to humans) secure 
grasps. These would be uncomfortable for a human to use but are perfectly acceptable 
for the robot. [Brock 87] has developed a module which permits executing motions using 
controlled slip within the hand. His analysis shows that by modulating the internal forces 
in an grasp, the grasp constraint (or freedom) may be altered in a predictable way. By 
partially relaxing the grasp and then by imposing forces with other fingers, objects or 
gravity, a grasped object can be made to move under controlled slip conditions. Repeated 
180 degree rotations of a grasped block using gravity have been demonstrated as well as 
the use of one finger to rotate an object held between the other two fingers. 

The stiffness control system was originally designed to impart controlled stiffness to 
grasped objects as might be required to place a peg in a hole or push an object into 
alignment. We have found that it also permits construction of stiffness that aid in the 
acquisition of objects. One particularly useful and simple construct we have employed to 
construct joint space stiffness matrices makes use of the fact that a matrix of rank n may 
be spectrally decomposed into a weighted sum of n unitary matrices. For example, if K is 
a matrix of rank 9 with nine linearly independent unit eigenvectors, fc„ and eigenvalues, 
A,, then 

k = 52 *. * 

If K is our joint space stiffness matrix then the elements of K represent the rate at which 
the torque increases on the tth- joint for a given deflection in the j th joint. The principal 
stiffness “direct ions”are defined by each k t . In this case the direction may be simply the 
motion of a particular joint ( k t has one non-zero element) or it may be a combination 
of joint motions. By choosing directions along which we wish to resist motion, k t , and 
magnitudes of stiffness along those directions, A,, we can construct K from the above 
relationship. If a set of directions is chosen which do not span the full space of motion, 
there will not be resistance to motion in the unspanned direction. 

It is also possible to use the reverse of this process to remove certain stiffnesses from 
a grasp. For example to permit the two fingers to balance the forces on them against the 
thumb, we would want to remove stiffness for differential motion of the fingers (here we 
mean moving finger 1 up while finger 2 moves down). By taking a nominal joint space 
stiffness, A', of full rank (for example, a diagonal matrix) and subtracting the matrix 
KiiiJ from it we reduce the rank of K by one. (This is strictly true only if xi is 
an eignevector of K, but often close enough.) If xj is a vector representing the desired 
differential motion of the fingers then the reduction of K's rank corresponds to the freedom 
introduced between finger 1 and 2. In general we can construct a new reduced rank stiffness 
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matrix by removing the stiffnesses from the “directions” in which we require compliance: 

Anew = A’ — A xi x f — A X2X 2 “ 

Current experimentation is concerned with integrating Brock’s slip analysis with a 
controlled slip planning module. In the near future the hand will be mounted on an 
arm for use in developing coordinated hand and arm operation. It appears that with a 
contact resolving fingertip sensor on each finger we will be able to implement an online 
grasp force controller which will seek to maximize structural restraint in the grasp. The 
same sensor will be used to detect incipient slip conditions by constant monitoring of 
normal and tangential force ratios. While all the above capabilities were designed for 
autonomous operation, it is clear that they can easily be integrated with some real time 
human intervention. It is easy to imagine master generated trajectories superimposed 
upon controlled grasp stiffness and condition monitoring. Significantly more difficult will 
be the problem of reflecting the force and kinesthetic information of grasping and inter-digit 
manipulation. 

Conclusions 

We have attempted to give a brief view of some of the mechanical and control character- 
istics upon which increased telerobotic capacity depends. It is clear that the successful 
cooperation of human and machine capabilities in hand control will depend both upon 
advancements in autonomous robot control and in the mechanics of the human/machine 
interface. The degree of complexity of such systems which will be both useful and tolerable 
may be ultimately best determined as we gain mort >erience with their use. 
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Figure 2. System Architecture, 







